联合学习（FL）使移动设备能够在保留本地数据的同时协作学习共享的预测模型。但是，实际上在移动设备上部署FL存在两个主要的研究挑战：（i）频繁的无线梯度更新v.s.频谱资源有限，以及（ii）培训期间渴望的FL通信和本地计算V.S.电池约束的移动设备。为了应对这些挑战，在本文中，我们提出了一种新型的多位空天空计算（MAIRCOMP）方法，用于FL中本地模型更新的频谱有效聚合，并进一步介绍用于移动的能源有效的FL设计设备。具体而言，高精度数字调制方案是在MAIRCOMP中设计和合并的，允许移动设备同时在多访问通道中同时在所选位置上传模型更新。此外，我们理论上分析了FL算法的收敛性。在FL收敛分析的指导下，我们制定了联合传输概率和局部计算控制优化，旨在最大程度地减少FL移动设备的总体能源消耗（即迭代局部计算 +多轮通信）。广泛的仿真结果表明，我们提出的方案在频谱利用率，能源效率和学习准确性方面优于现有计划。

基于Eikonal方程的最小测地模型能够在各种图像分割场景中找到合适的解决方案。现有的基于测地的分割方法通常与几何正则化术语一起利用图像特征，例如欧几里德曲线长度或曲率惩罚长度，用于计算测地曲线。在本文中，我们考虑了一个更复杂的问题：在先前用凸形形状找到曲率惩罚的测距路径。我们建立了依赖于取向升降策略的新测地模型，通过该曲线可以映射到高维定向依赖的空间。凸起形状以前用于构建编码特定曲率约束的局部测地度量的约束。然后，可以通过最先进的Hamiltonian快速行进方法有效地计算定向空间中的测地距离和相应的闭合大气路。此外，我们将所提出的测地模型应用于活动轮廓，导致有效的交互式图像分割算法，其保留凸起形状的优点和曲率损失。

联合学习（FL）是一个带有边缘计算的充填地的新兴分布式机器学习范式，是具有在移动边缘设备上具有新颖应用的有前途的区域。在FL中，由于移动设备通过共享模型更新，因此在中央服务器的协调下基于其自身的数据进行组合培训模型，培训数据保持私密。但是，在没有数据的核心可用性的情况下，计算节点需要经常传送模型更新以获得汇聚。因此，本地计算时间与将本地模型更新一起创建本地模型更新以及从服务器发送到服务器的时间导致总时间的延迟。此外，不可靠的网络连接可以妨碍这些更新的有效通信。为了解决这些问题，我们提出了一个延迟有效的流动机制，可以减少模型融合所需的总时间（包括计算和通信延迟）和通信轮。探索各种参数对延迟的影响，我们寻求平衡无线通信（谈话）和本地计算之间的权衡（为工作）。我们与整体时间作为优化问题制定了关系，并通过广泛的模拟展示了我们方法的功效。

管状结构跟踪是计算机视觉和医学图像分析领域的关键任务。基于最小的路径的方法在跟踪管状结构中表现出具有强的能力，通过该方法可以自然地建模，作为用合适的测地度量计算的最小测地路径。然而，现有的基于路径的基于路径的跟踪方法仍然遭受诸如快捷方式和短分支组合问题的困难，特别是在处理涉及复杂的管状树结构或背景的图像时。在本文中，我们介绍了一种新的最小路径基于基于路径的基于型号，用于尽可能多的交互管结构中心线提取与感知分组方案。基本上，我们考虑了规定的管状轨迹和曲率惩罚的测地路，以寻求合适的最短路径。所提出的方法可以从管状结构上的局部平​​滑度和基于使用的图形的路径搜索方案的全球最优性中受益。合成和实图像的实验结果证明，该模型确实获得了与最新的基于路径的管状结构跟踪算法比较的优惠。

Purpose: Tracking the 3D motion of the surgical tool and the patient anatomy is a fundamental requirement for computer-assisted skull-base surgery. The estimated motion can be used both for intra-operative guidance and for downstream skill analysis. Recovering such motion solely from surgical videos is desirable, as it is compliant with current clinical workflows and instrumentation. Methods: We present Tracker of Anatomy and Tool (TAToo). TAToo jointly tracks the rigid 3D motion of patient skull and surgical drill from stereo microscopic videos. TAToo estimates motion via an iterative optimization process in an end-to-end differentiable form. For robust tracking performance, TAToo adopts a probabilistic formulation and enforces geometric constraints on the object level. Results: We validate TAToo on both simulation data, where ground truth motion is available, as well as on anthropomorphic phantom data, where optical tracking provides a strong baseline. We report sub-millimeter and millimeter inter-frame tracking accuracy for skull and drill, respectively, with rotation errors below 1{\deg}. We further illustrate how TAToo may be used in a surgical navigation setting. Conclusion: We present TAToo, which simultaneously tracks the surgical tool and the patient anatomy in skull-base surgery. TAToo directly predicts the motion from surgical videos, without the need of any markers. Our results show that the performance of TAToo compares favorably to competing approaches. Future work will include fine-tuning of our depth network to reach a 1 mm clinical accuracy goal desired for surgical applications in the skull base.

Nowadays, fake news easily propagates through online social networks and becomes a grand threat to individuals and society. Assessing the authenticity of news is challenging due to its elaborately fabricated contents, making it difficult to obtain large-scale annotations for fake news data. Due to such data scarcity issues, detecting fake news tends to fail and overfit in the supervised setting. Recently, graph neural networks (GNNs) have been adopted to leverage the richer relational information among both labeled and unlabeled instances. Despite their promising results, they are inherently focused on pairwise relations between news, which can limit the expressive power for capturing fake news that spreads in a group-level. For example, detecting fake news can be more effective when we better understand relations between news pieces shared among susceptible users. To address those issues, we propose to leverage a hypergraph to represent group-wise interaction among news, while focusing on important news relations with its dual-level attention mechanism. Experiments based on two benchmark datasets show that our approach yields remarkable performance and maintains the high performance even with a small subset of labeled news data.

Quantum machine learning (QML) has received increasing attention due to its potential to outperform classical machine learning methods in various problems. A subclass of QML methods is quantum generative adversarial networks (QGANs) which have been studied as a quantum counterpart of classical GANs widely used in image manipulation and generation tasks. The existing work on QGANs is still limited to small-scale proof-of-concept examples based on images with significant down-scaling. Here we integrate classical and quantum techniques to propose a new hybrid quantum-classical GAN framework. We demonstrate its superior learning capabilities by generating $28 \times 28$ pixels grey-scale images without dimensionality reduction or classical pre/post-processing on multiple classes of the standard MNIST and Fashion MNIST datasets, which achieves comparable results to classical frameworks with 3 orders of magnitude less trainable generator parameters. To gain further insight into the working of our hybrid approach, we systematically explore the impact of its parameter space by varying the number of qubits, the size of image patches, the number of layers in the generator, the shape of the patches and the choice of prior distribution. Our results show that increasing the quantum generator size generally improves the learning capability of the network. The developed framework provides a foundation for future design of QGANs with optimal parameter set tailored for complex image generation tasks.

Recent advances in neural radiance fields have enabled the high-fidelity 3D reconstruction of complex scenes for novel view synthesis. However, it remains underexplored how the appearance of such representations can be efficiently edited while maintaining photorealism. In this work, we present PaletteNeRF, a novel method for photorealistic appearance editing of neural radiance fields (NeRF) based on 3D color decomposition. Our method decomposes the appearance of each 3D point into a linear combination of palette-based bases (i.e., 3D segmentations defined by a group of NeRF-type functions) that are shared across the scene. While our palette-based bases are view-independent, we also predict a view-dependent function to capture the color residual (e.g., specular shading). During training, we jointly optimize the basis functions and the color palettes, and we also introduce novel regularizers to encourage the spatial coherence of the decomposition. Our method allows users to efficiently edit the appearance of the 3D scene by modifying the color palettes. We also extend our framework with compressed semantic features for semantic-aware appearance editing. We demonstrate that our technique is superior to baseline methods both quantitatively and qualitatively for appearance editing of complex real-world scenes.

Functionality and dialogue experience are two important factors of task-oriented dialogue systems. Conventional approaches with closed schema (e.g., conversational semantic parsing) often fail as both the functionality and dialogue experience are strongly constrained by the underlying schema. We introduce a new paradigm for task-oriented dialogue - Dialog2API - to greatly expand the functionality and provide seamless dialogue experience. The conversational model interacts with the environment by generating and executing programs triggering a set of pre-defined APIs. The model also manages the dialogue policy and interact with the user through generating appropriate natural language responses. By allowing generating free-form programs, Dialog2API supports composite goals by combining different APIs, whereas unrestricted program revision provides natural and robust dialogue experience. To facilitate Dialog2API, the core model is provided with API documents, an execution environment and optionally some example dialogues annotated with programs. We propose an approach tailored for the Dialog2API, where the dialogue states are represented by a stack of programs, with most recently mentioned program on the top of the stack. Dialog2API can work with many application scenarios such as software automation and customer service. In this paper, we construct a dataset for AWS S3 APIs and present evaluation results of in-context learning baselines.

Vision transformers (ViTs) are quickly becoming the de-facto architecture for computer vision, yet we understand very little about why they work and what they learn. While existing studies visually analyze the mechanisms of convolutional neural networks, an analogous exploration of ViTs remains challenging. In this paper, we first address the obstacles to performing visualizations on ViTs. Assisted by these solutions, we observe that neurons in ViTs trained with language model supervision (e.g., CLIP) are activated by semantic concepts rather than visual features. We also explore the underlying differences between ViTs and CNNs, and we find that transformers detect image background features, just like their convolutional counterparts, but their predictions depend far less on high-frequency information. On the other hand, both architecture types behave similarly in the way features progress from abstract patterns in early layers to concrete objects in late layers. In addition, we show that ViTs maintain spatial information in all layers except the final layer. In contrast to previous works, we show that the last layer most likely discards the spatial information and behaves as a learned global pooling operation. Finally, we conduct large-scale visualizations on a wide range of ViT variants, including DeiT, CoaT, ConViT, PiT, Swin, and Twin, to validate the effectiveness of our method.